Unspecified distribution in single disorder 

problem 



Wojciech Sarnowski a '* Krzysztof Szajowski a 

' Wroclaw University of Technology, Institute of Mathematics and Computer 
Science, Wybrzeze Wyspianskiego 27, 50-370 Wroclaw, Poland 



Abstract 

We register a stochastic sequence affected by one disorder. Monitoring of the se- 
quence is made in the circumstances when not full information about distributions 
before and after the change is available. The initial problem of disorder detection is 
transformed to optimal stopping of observed sequence. Formula for optimal decision 
functions is derived. 
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1 Introduction 



The paper is focused on sequential detection using Bayesian approach. Disorder 
problem in this framework was formulated by A.N. Kolmogorov at the end of 50's of 
previous century and solved by |Shiryaev(1961)| . The next turning point is paper by 
Peskir and Shiryaev(2002)] where authors provide complete solution of basic prob- 



lem. From this time many publications provide new solutions and generalizations 
in the area of sequential detection. Some of them are articles by Karatzas(2003)| 



and Bayraktar et al.(2005)Bayraktar, Dayanik, and Karatzas . For discrete time 



case there are some detailed analysis in the papers by Bojdecki and Hosza(1984j| , 
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[Moustakides(1998)| , [Yakir(1994)] , [Yoshida(1983)1 , [Szajowski(1996)| and the 



papers cited there. 



Such model of data appears in many practical problems of the quality control (see 
Brodsky and Darkhovsky Brodsky and Darkhovsky(1993)| , Shewhart |Shewhart(1931)] 



and in the collection of the papers Basseville and Benveniste(1986)j ), traffic anoma- 



lies in networks (in papers by Dube and Mazumdar [Dube and Mazumdar(2001)| , 
Tartakovsky et al. [Tartakovsky et al.(2006)Tartakovsky, Rozovskii, Blazek, and Kirn] ), 
epidemiology models (see Baron [Baron(2004)] ). In management of manufacture it 
happens that the plants which produce some details changes their parameters. It 
makes that the details change their quality. The aim is to recognize the moments of 
these changes as soon as possible. 

This paper focuses attention on models under assumption of uncertainty about dis- 
tribution before or after the change. The example of such models can be found in 
research by Dube and Mazumdar(200T)] with application to detection of traffic 



anomalies in networks or in paper by [Sarnowski and Szajowski(2008)| . The solu- 
tion of a single disorder model with unspecified distribution of observed sequence is 
presented. Section [2] specifies the details of investigated model. The transformation 
of the optimization job to the optimal stopping problem for the specific stochastic 
process is considered in Section [3j A construction of the optimal estimator of the 
disorder moment is given in Section UJ Technical parts of investigations are moved 
to Appendix. 



2 Description of the model 

2.1 Basic notations 

For further considerations it will be convenient to introduce the following notation 
which will make our formulas more compact and clear 

—k,n — {-Eh ' "^fc+l j ■••) %n—l j %n)i k 71, 
n—m n 

r=k+l r=n—m+l 

A kn = A k x A k+ i x ...xA n , 
where: n^mi u r = ^ for fni > ^2 and u r 6 5ft, A4 G B, k < i < n. 

It will be convenient to write = (J3i, fo) and denote by a = (an, . . . , an x , ■ ■ ■ , a/ i, ■ ■ ■ , ct^^) 
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any matrix Iq x l\: 



a n ai2 ■■■ a lh 

&l i a io2 • • • a l h 
In consequence vectors W, b, p represent: 

7f = (TTll , . . . , TTu 2 , . . . , 7T{ 1, . . . , 7T/ oil ) 
& = • • • ) blhi ■■■ 1 bi i, • • • , fejo/j) 

P= Oil, • • • .Plix, • • • ,PZ 1, • • • 

We need also notation for vector of densities fx ,l (y). Let f%(y), where x,y € E 
stands behind: 



Moreover let us introduce operation "o". For vectors a and (3 we put: 

a o (3 = (a n (3 n , ■ ■ ■ ,aih0Ui,- ■ ■ ,aj iA i> • • • , a i hPioh) ■ 
2.2 Change point problem 

Let (X n ) ng N be sequence of observable random variables defined on (fi, J 7 , P) with 
value in (E, £>), EcS. Sequence (X n ) generates filtration T n = a(Xo,Xi, ...,X n ). 
On the same space there are also defined variables 6, (3\ and 02- takes values 
in {1,2,3,...}. Variables (3±, 02 are valued in Ik = {1,2,...,^}, where Ik G N, 
k = 0, 1. Let us assume the following parametrization: 



/!?(//) (./:!'•' i//* ./:: u (/;)••• 

» ' 



f° x > l °(y),...,f° x > l °(y)). 
' 



Zi times 



ii times 



i,/3 2 



if n = 1 



P(0 = n\fc 



i,@2 



J 



< 



(1 - TTi^Pij 2 qij, if n > 1, 
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where i G I ,j G Ji, Z)ie/ ,ieii & y = 1 ' - °> G I ' ^ P*J = 1 ~ ?*J G (°' X )- We 
have 

^^P(fl = Mi = i,A = j) = i 

fc=l i£lo j£h 

The change of the conditional densities in random moment 9 is investigate in this 
model. The transfer between distribution is described by conditional probabilities 
bij = P(/?2 = j\(3i = i). For completeness it will be assumed that the state of (3\ is 
stable before 9 and the same as at the moment 0. The marginal distribution of 9 
has a form 

P(0 = k) = P(0 = k, Pi = i, h = j) 

hi 

yii.j~i.i- 1 >; * if/c = i, 

= < 

The observed sequence has a form 



X n - X^ 1 ■ l {e>n , /?!=»} + X n * ■ l {e< n , p 2 =j ^I'i^xltxY W 

where (X% 1 , 5n' ! ,P?), r = 0, 1, are Markov processes and cr-fields: Q n ' 1 = g(Xq 1 , . . . , Xn*), 
with i G Jo, J G /i, r = 0, 1 and n G {0, 1, 2, . . .}. Variables 9, Pi and /?2 are not 
measurable w.r.t T n . 

On the space (E, B) there are cr-additive measures //(•) and measures /i*'* absolutely 
continuous with respect to \i. It is assumed that the measures Pi'('), i = 1,2, ... 
k = 0, 1, have following representation: 



P*'*({ W : X** G fl}) =P(Xf' < G = s) = / f*>\yMdy) 

Jb 

= I i x k /{dy) = l i k /{B). 

JB 

for any B G 5. The conditional densities fx' 1 ^), ■ ■ ■ ,fx' lk (-) are different and sup- 
ports of all measures fix are there same for given x G E. It is the model of the follow- 
ing random phenomenon. At the beginning we register process {Xn % ,n G N}, where 
i G Iq is unknown. At random moment 9 initial process is switched on {Xn 3 , n G N} 
where j G I\ is unknown. It can be interpreted as disorder of {X n ,n G N} causing 
change in distribution of {A n } ne ^. We monitor the process and we wish to detect 
the change as close 9 as possible. However our knowledge about densities before and 
after the change moment 9 is limited generally to the information about sets of pos- 
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sible conditional densities only: {fx' l {y),i G Io} and {/i J (y),J £ h} respectively. 
We also know probabilities of distribution pairs bij and parameters iiij. 

For i 6 /oiJ S A let us introduce functions W 1 ' 3 , <^ 4J , A 1 ' 3 , A 1 ' 3 denned on the 
product N x (x -j^E) x [0, 1] with values in 



k=0 
I 



I k=l 



+ a^|+i(5o,i+l) ( 2 ) 
+ al# 1 (x ,i+ 1 ), (3) 



^ M (^o,/+i,a) = ^ J (^o,z+i, «) ~ (1 - «)p« ^fey+i)' 



Next let us define on N x (x.^ 2 E) x (x^JO, 1]) x (x££[0, 1]) function 5, 5 



So, fc+1 , 7, <*) = H 7ij So,fc+i, * 



(4) 
(5) 



For any D n = {w : X, G -Bj, z = 1,2,..., n}, where Bi E B and any x G E 
define: 

P x {D n ) = P(D n \X = x)= S(n - l,x , ^Tf^fe ) 



For the process ([T]) the set of estimators for the disorder moment 9 is & x - 
the set of stopping times with respect to {^njneNu-jo}- The construction of the 
optimal estimator is to find a stopping time r* G (5 X such that for any iGE 



P*(|0-t*| <<£)= sup P^-rl <d), 



(6) 



where d G {0,1,2,...} is fixed level of detection precision. 



3 Existence of solution 



In this section we are going to show that there exists solution of the problem 
flEJ). Let us define: 
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Z n = P(\6-n\ <d\F n ), n= 1,2,..., 

K = ess sup P(|6» - n| < d | .F n ), n = 0,l,2,... 

{rGS x , r>n} 

r = inf{n : Z n = V n } (7) 

Notice that, if = 0, then Z T = P(\6 — r| < d | jF r )for r £ (5 X . Because 
•7"n Q Ft (when n < r), we obtain 

V n = esssupP(|# — r| < d | jF n ) = ess sup E(E(I{| 6) _ r |< d }|jF r ) | jF n ) 

r>n r>n 

= esssupE(Z r I jF n ) 

r>n 

The following lemma states that solution exists. 

Lemma 1 Stopping time Tq given by |7|) is a solution of the problem 

PROOF. Applying Theorem 1 from [Bojdecki(1979)j it is enough to show 
that lim Z n = 0. For all n, k, where n > k we have: 

n— >oo 

Z n = E(I{|<9_„|<4 | Fn) < E(supl{| e _j|< (i } I F n ) 

Basing on Levy's theorem we get lim sup,^^ Z„ < E(sup J>fc I{\g-j\<d} \ J-'oo) 
where = a (U$£Li F n )- We have: limsup J>fcj ^-{\e~j\<d} = a.s. Basing 
on dominated convergence theorem we get we state that 

lim E(supl{|0_j|<d} | JFoo) = a.s. 

what ends the proof. 

It turns out that we need at least d observations to detect disorder in optimal 
way: 

Lemma 2 Let r be stopping rule in the problem (TJJ). Then rule f = max(r, d+ 
1) is at least as good as r (in the sense of 

PROOF. For r > d + 1 the rules are the same. Let us consider case when 
r < d + 1. Then f — d + 1 and: 
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P(|0 - r\ < d) = P(r - d < 9 < t + d) = P(l < 9 < r + d) 

<P(1 < 9 < 2d+ 1) = P(f-d< 9 < t + d) = P(|0-f| < d). 



4 Construction of the disorder moment estimator 

4-1 Function and processes 

Let us fix parameters W, b and set initial state of X n : P(Xo = x) = 1. We 
denote ip = (w, b, x) and we will write P ¥> («) to emphasis that the probability of 
the events defined by the process are dependent on this a priori set parameters. 
Let us define the following crucial posterior processes: 



P v (9 < n\F% 



(9) 



where n E N, i E I 0l j E h, = o-{T n , I{/3=(ij)})- Process is designed 
for updating information about disorder distribution. B 1 ^ in turn refreshes 
information about distributions of variables Pi, P2- Notice that IT^ 7 , B 1 ^ starts 
from following states: Hq J = 0, Bq 3 = by Dynamics of IT^ and B 1 ^ 3 are 
characterized by formulas ( 1A.10I) . ( 1A.11I) . The above notations hold also for 
(El), (ED: 



n = ii' II' 2 n 1 ' n 



B n ^[B^,...,B^,...,B^,...,B^). 

At the end of section let us define auxiliary functions IT V (-,-,-), r v (-,-,-, 
For x,y EE, a, jij, ^ G [0, 1], % E h, j E h put: 



ihh 



n JJ (A;,x 0in ,a) 



'y ij ^ l ' :i (k,x 0tn ,5i j ) 
S{k,x S)yn ,^, 8) 



(10) 
(11) 



Let D n = {oj : 2Lo,n e iio,n}; X = x and Bi E B. We have 
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p*(e>n,p = (i,j),D n ) 



(i - i$WdP* 



Or: 



where 



(12) 



Sn J (x n ) — TTijLlf (x n ) + (1 — Kij)Pij Lq^Xq^ 

n 



(13) 



and iS n (x 0)n ) — &j_ 7 -iS'^ J (x 0iTl ) — S(n l,XQ^ n ,b, 



7T . 



5 Solution 



According to Shiryayev's methodology (see |Shiryaev(1978)| ) we are going 
to find solution reducing initial problem (jfJJ) to the case of stopping Random 
Markov Function with special payoff function. This will be done using posterior 
processes flHD-©- 



Lemma 3 For n > d + 1 



P v {\9-n\ < d) 



K2L n -l-d,n, n n, B n ) , if 71 > d + 1 



if n — d+ 1 . 



where 



d+l ri,j 



h(x l4+2 , 7, 5) = J2 1 - Py + %■ Z! 



d+2/ 



i -J 



k=l Pij Lq (2Ll,d+2j 

%,3)=£(i-pJ(i-7«))fy, 



(1 - 7ij)*ij> 



(14) 



(15) 
(16) 



xi, ...,x d+2 e E, 7y, 5y e [o, l], z e J , j e A. 
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PROOF. Let us rewrite initial criterion as expectation: 



P v (|0 - n\ < d) = W \P*(\e - n\ < d \ F n )\ . (17) 



Let us analyze conditional probability under expectation in equation (fT7j) using 
total probability formula 



( \9 - n\ < d | F n ) = P v (e <n + d\F n )- P v (9 < n - d - 1 | F n ) (11 

hi i,j 

because 



P^(9 <n + d\F n )=W(l e < n+d \T n ) = Y J V ip (he<n+d}hp=(i, j )}\J r n) 

y 

= £ E^(I {B < n+d} I {i=iid)} \P^) \F n ) 

hi 

= £ W(l {i={iJ)} W(I {e < n+d} \F n ) 
hi 

= ^W(I {e < n+d} \J r l; 3 )W(I { p ={i j ) - } \J r n ). 



The last equality is a consequence of the very special form of the extended 
(T-field T]iK The random variable measurable with respect to T 1 ^ is also T n 
measurable. Putting n = d+ 1 in Lemma El we get P^iO < n — d — 1 | T n , (3 = 
= 0, for i G Io,j G h- Hence 

P*(|0 -n\<d\F n ) = Y, <n + d\ F n )P*(£ = (i, j) I T n ). 

hi 

Lemma|limpliesthatP v (|6>-n| < d) = W \h(Ji d+1 , B d+1 )] . Now let n > d+1. 

Basing on Lemma |4] probability P v (# < n + d \ J- n ) is given by (1A.3|) . From 
Lemma El we know that P^id < n — d— 1 | jF n ) is expressed by equation ( 1 A. 71) . 
Formula ( 1A.7I) reveals connection between payoff function (|T4l) and posterior 
process at instants n and n—d—1, i.e IT^ 7 , n^i d _ 1 for % G io, j £ A- Dependence 
on can be rule out by expressing T^^-d-x m terms of H^f . By Lemma [6] 

and (1A.9I) we get 
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ll n-d-l 



k=0 

d 

(1 - Un j )(qij^pf^ k L^ J +1 (X_ n _ d _ ln ) - L d d +1 (x_ n _ d _ ltn 

k=0 

lL n Pij ^0 \^-n-d-l,n 



(19) 



The result (|T9l) and formula (1A.7|) lead us to: 



)<n-d-l | F n ,0 = (i,j)) (20) 

P^ fl -^0 J C^n-<i-l,n)n^ 3 ' ~~ lij J2k=oPij ^fc+l (2Ln-d-l,n) (1 ~~ 



Applying equations ( 1A.3I) and ( 1201) in formula ( jTSl) we get the thesis. 



Notice that for n > d + 1 function h under expectation in (TH1) depends on 
process = (2C n „ d _ l n , II n , £?„). It turns out that {r/ n } is Markov Random 
Function (see Lemma M in Appendix lAl) . We do not care about {r] n } for n < 
d+1. It is a consequence of discussion in Lemma[2]which leads to the conclusion 
that under the considered payoff function (criterion) it is not optimal to stop 
before instant d+1. The decision maker can start his decision based on at 
least d+1 observations Xi, . . . , X d+ i. 

Lemmata [3] and [8] imply that initial problem can be reduced to the optimal 
stopping of Markov Random Function (r] n , T n , Pp^L 1; where y = (x n _ d _ l n , j, 5) G 
E = E d+2 x [0, l] hh x [0, l] hh with payoff described by (EGO). However, the new 
problem is no longer homogeneous one as it is emphasized by the definition 
of y. It is a consequence of the fact that the process {r] n } for n < d + 1 has 
formally different structure than for n > d + 1. Thus, the payoffs for instances 
n < d + 1 are different. Lemma [8] gives a justification to work with the ho- 
mogeneous part of the process in construction the optimal estimator of the 
disorder moment. 

To solve the maximization problem fTT^l) . for any Borel function u : E — > 3? 
let us define operators: 
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Tu(x M+2 ,7,£)=E^ - 



u (2Ln-d,n+l) n n+ i, B n+ i t 
u [2Ln-d,n+l> n n +l, B n+ i) | [X_ n _ d _ ln: Yi n , B n ) 



(£l,d+2,7><5) 



Qw(si, d+ 2, 7, S)=max.{u{x x4+2 , 7, 5), T«(x M+2 , 7, 5)}. 



Operators T and Q act on function h and they determine the shape of optimal 
st opping rule r*. Recursive formulas are given by Lemma EE which is presented 
in Appendix |A] Lemma [9] characterizes structure of sequence of functions 
s k(Z£i,d+2) 7) where x G E d+2 , 7, <5 G [0, l] /l?2 , which is used in the theorem 
stated below. 

Theorem 1 The solution of problem (0|) is the following stopping rule: 



inf { n > d + 2 : (X^^, II n , B n ) G D*} , if h(U d+1 , B d+1 ) < s*(X 14+2 , U d+2 , B d+2 ), 

(21) 

i//i(n d+ i,5 d+ i) > s*(X 1>d+2 ,U d+2 ,B d+2 ), 

where the stopping area D* : 



D* = { (x_ M+2 ,7,<5) G S* : h(x ljd+2 , 7, 5) > s*(x M+2 , 7, 5) \, 



and s*(x ljd+2 , 7, 5) = lim^oo s fc (x M+2 , 7, S) . 



PROOF. First let us consider subproblem of finding the optimal rule r* G 

$d+2 : 



E^ 



M2^-d-i,^n~*,£?~0 = sup W h{X T _ d _ ljT ,U T ,B T ) 



(22) 



Then, basing on Lemmata (TJ [2] and according to optimal stopping theory (see 
|Shiryaev(1978)1 ) it is known that r defined by ([7]) can be expressed as 

r = inf{n > d + 2 : /i(2C n -i-d,n, n n , B n ) > h*{X n _ x _ dtTl , n n , J3 n )}, 

where h*{x l d+2 , 7, 5) = linn. >oc Q fc ^(x 1(i+2 , 7, 8). The limit exists according 

to the Lebesgue's theorem and structure of functions h and s^- Lemma [9] 
implies that: 
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r = inf {n > d + 2 : h{X n _ x _ d ^ II n , B n ) 

> max {/i(X„._i-d,n 5 TT n , 5 n ) , s*(2Cn-i-<i,n> TT n , B n ) } } 

= inf {fi > d+ 2 : ftQ£n-l-d,f» n n> > s *G£n-l-<f,n, n n, B n )j . 



According to optimality principle rule f* solves maximization problem of (fl4j) 
if only at n = d + 1 the payoff /i will be smaller than expected payoff in 
successive periods (for n > d+ 1). Thus, another words: 

r* = r*, if h(X 04+1 , n d+1 , S d+1 ) < s*(X M+2 , n d+2 , S d+2 ). (23) 
In opposite case r* = + 1. This ends the proof of formula (I2ip . 
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A Lemmata 



In appendix we present useful formulae and lemmata which help to obtain 
solution of problem (Ej) . 

Remark 1 For n > I > 0, k > 0, i G Iq, j G l\, on the set {u : 2Lo,i 
A ;,t4o = {a;}, Ai G jFj,z < £} f/ie following equations hold: 



( 6 = n + k \ A 0>/ G Ay , £ = (2, j),9> n) 

if n,k > 0, 



Pij Qij 5 



7T 



if n = 0, k = 1, 



(1 - Kij)v% 2 qij, ifn = 0,k > I, 



Remark 2 (1) The simple consequence of the formula ( L4.ll) we get 
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W {6>n + k\ X 0il G Aojl, §_ = (i, j),9> n) 
Pij, ifn,k>0, 



(l-7Ty)p*-\ ifn = 0,k>0. 



(A.l) 



Formula hA.l\) for k = 1 is given by: 

= P*(6 >n + l\ X j G Aj^P = (i, j),9> n) 
Pij, tfn>0, 
(l-7Ty), ifn = 0. 

Lemma 4 For n > 0, k>0,iElo,jE I\ the following equation is satisfied: 



(A.2) 



P*{6 < n + k | F n ) = 1 - 4(1 - (A.3) 
PROOF. We are going to show equality on the set {a; : X_ n G -B n , 5 = 

P*(0 > n + fc, X ,n e Bo in , = (i, j))= J I w .e >n+ k}dP v (A.4) 

{wJfo,„^o,„,^=(W)} 



E^(I {a):0>n+fe} |^ re )dP^ 

{w:2[o,neBo,».£=(ij)} 

= | E^(I { ^ =(iJ)} E^(I {a;:e>n+fe} |J n )|^ ri )dP^ 
By direct computation calculation we get 

P^(6 >n + k, X ,n G £o,n, £ = (z, j)) (A.5) 

„ oo 

=Pij / ij, x 1 ^7 T^Uo,» W^,J 

{w:2fo,neBo,»} {w:2fo,»eBo,n} 

=P«/ P ^ = fr-fll-WC* > n\T n )dP* (A.6) 

{w:2Q,, n eBo,n} 
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Henceforth we have 



E^(I { ^ =(ij)} E^(I { ^>^ > n\T n ). 

Comparison of (IA.4[) and (IA.5[) implies IA.3I and this ends the proof of lemma. 



Lemma 5 For n > k > 0, i £ Iq, j £ I\ it is true that 



P *(0<n-fc-i|^„) = i-(i-i#) L /Jf::: s+1 ' n) ) 



s=l Pij-^Q 3 'QLn-s+l,r 



PROOF. If n = k + 1 then 



P v (0 <n-k-l\F n ) = P^fl < | F k+1 ) 



Because of the fact that 9 > a.s.: 



(A.8) 



Hence formula ( 1A.7I) holds. The case where n > k + 1 we have 

fe+i 

P v (6> > n - k - 1 | P n ) = W(6 >n\F n ) + J2 p ^(# = n-s\F n ). 

8=1 

On the set D n = {u : = (i,j),2Lo, n £ iio,n> -^o = {^j} we have 



n - s,D n ) 



D, 



I {e=n _ s} cZP ¥! 



p^(# = n - s|jF n )dP* 



q.ijLsji(x n -s,n) (1 ~ Kj^Pjj l bjjLQ (x$ n ) S^(x Qn ) 

<Sn(jiO,n) 



git (2Ln-s,n) Sn\XjO, r , 
n—s,n 



(i - nswdp* 



Therefore 



Pij A) (2Cn-s,n) 
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and 



> n — k — l \F n ) = 1 + qij 



s=Q fij 



Lemma 6 For n > I > 0, i £ Iq, j G I\ following equation holds: 



U i >i(l,X n _ l _i n ,I^_ l _ 1 ),ifn> l + l, 



ir ,J = 4 

n « 



if n = 1+1. 



(A.9) 



Remark 3 In particular, taking 1 = 0, we get equation characterizing "one- 
step" dynamics of the process U 1 ^ 3 : 



n 



/i„_ 1 (^»)(9«^« ff »-i)+/x;_ 1 ( x »)py( 1 - n ;-i) 

/i'(^l)^+/xn(^l)( 1 -^)' 



ifn>l, 
ifn = 1, 



(A.10) 



wi£/i initial condition Uq 3 = 0. 



Remark 4 For / > recursive structure defined in equation liA.ty) requires 
vector of initial states Hq 3 , IL l { 3 , . . . , IL l '- ? . S'tote n J is given above. To obtain 
remaining states l¥{ 3 , . . . , II]' 3 it is enough to apply formula liA.10\) . 



PROOF, (of Lemma [6]) Condition IIo = has been shown in lemma [5] 
(equation (1A.8|1 ). We have following recursive relation; 
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Sn-s-l(Zk>,n-s-l) G^n-s-l.n; H M (n S,X n _ s _ l n , TTij)) 

= S^ J _ s _ 1 {x 0n _ s _ 1 )Il 1 ! l :l _ s _ 1 Li +1 (x n _ s _ ltn ) + 5 , n _ s _i(x 0n _ s _ 1 )(l — II^ S _ 1 ) 



-n—s—l,n) 



k=0 



n— s— 1 



""ij^nisfeo^-s-l) + (1 ~ n ij)Qij X ^n-s-fcQzio.n-s-l) I -^s+lfe 



n—s—l,n) 



k=l 



+ (l-Tv)P« S lL o(^0,n- S -l) XX fe %' L fc+l(^n~ S -l,n) +P?/ 1 ^0fe„.-s-l,n) 

\fe=0 

ra— s— 1 

TTii^feo.n) + (1 - Tij [ X] Pij~ 1( Hj L n-k+l(X 0tn ) 

k=l 

s 

k=0 

n— s— 1 

^ijLn J (x 0>n ) + (1 - 7Tij) [ ^ Pi^qijLn^k+liX^J 

k=l 

n 

+ X! /'o W- 



k=n— s 



n-fc+l(^0,n) +P^o(rKi 



0,nJ 



fc=l 



= KijL^{x 0;n ) + (1-TTij 

Now, on the set D n = {uj : X_ n 6 i?o n }, Xq = x and Bj G B we have by (fT2j) : 



P(e>n,^=(t,i),D n )= y I {0>n}( fP^ = y P*(0 > ra^)^ 

{^=(i,i),D„} {^=(i,j),D„} 



^J(n - S,I„_ s _ 1)n ,ny(ll - S.Sn-a-i^jTTij)) 

n 



^n-s-l(^0,n-s-l) 



Dn 



n— s— 1\— n— s— l,n) AX n— s— 1; 



This follows 



-^0 Q£n-s-l,n) 



n— s— IV— n— s— l,n> J - l n— s— 1/ 

In the case where n = s + 1 the proof is similar. 
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Lemma 7 For n > 0, i G Iq, j : G h we have 



T^^Xn^^Bn^Un^), if 71 > 1, 



B 



(A.n; 



^i 1 ^(o,x (U , 7 ^) 

S(0,X 01 ,b,K) ' 



if n = 1. 



witt condition Bq° = bij. 



PROOF. First, let us verify the initial condition: 



Let n > 1. Let us consider formula (1A.11I) on the set D n = {uj : X 0n G 
B 0n ; B = {x}, Bi G B for 1 < i < n}\ 



P^iP =(*,]), K , n eB 0!n )= [ l {p _ =m dW = [ W{I {p _ =m \T n )dP* 

J Dn J D n 

bijSn" 1 (2k),n) c , \.,(J \ 
q / \ >Jn{SjO,n) t L \ a 3Ll,n) 
B ln O n {X 0n ) 

WQCon) ^ (A12) 
D n O n (A_o,nJ 

Taking into account the formulae ffT3l) , ([2]) , (TJ]) and ffTTT) we have gotten flA.111) 
for n > 1. The case n = 1 is a consequence of (1A.12j) and (TTTj) with ([3]) and 



Lemma 8 Let rj n = (X_ n _ d _ ln ,Il n ,B n ), where n > d+1. System (?7 n ,JF n ,P,^ 
is Markov Random Function. 



PROOF. It is enough to show that n n +i is a function of rj n and variable X n+ i 
as well as that conditional distribution of J n+1 given T n depends only on r\ n 
see [Shiryaev(1978)l ). 

For xi, Xd+2i y £ E, 7^, <5jj G [0,1], i G ioi j £ A let us consider the following 
function 
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= (%2 ! d+2,y,' sll ' 1 (o,zd+2,y,Sii), ■ ■ • ,n 1,i2 (o,2: d+ 2,y,5i/ 2 ), 

U h ^(0, x d+2 , y,S kl ),..., n^ 2 (0, x d+2 , y, 6 hh ), 
V 1 ' 1 (O^^ty, 7,^), ■ ■ . 1 T 1 ' h (0,x d+2 ,y, 7 y,5),.. . , 
T' 1 - 1 (0, x d+2 ,y,i,5),...,r h > h (0, x d+2 , y, 7, 5)) . 



We will show that r] n+1 = Lp(r/ n ,X n+1 ). Using formulas flA.101) and flA.lip we 
express iV^+i as a function of H l rf and Bll J +l as a function B 1 ^ . Then: 



ip ( r? n ,X n+ i) 

= <P(2Ln-d-l,ni ^n, -B n , -Xra+l) 

= (1™-^! ^n+b n 1,1 ^, i„ it!+ i, ny ), . . . ,n 1,ia (o,Xn,n+i> n ri 
n /i ' 1 (o,x njn+1 ,n^ 1 ' 1 ), . . . ,tf i ' i2 (o,x nri+1 ,iiJ ) i ' /2 ), 
r 1,1 (o, n n ), . . . , t 1,12 (o, 2L nn+1 , B n ,n 

t 11 ' 1 (o, x_ n n+l , B n ,u n ) , . . . ,T li,t2 (o, x_ nn+1 , B n ,u 

= (2L n -d,n+li n n+ i, B n+ i) = T] n+ i. 



n /)■■■> 



^nyj • • • j 



Let us consider now the conditional expectation u(X n+ i) under the condition 
of a -field J-" n , for Borel function u : E — ► 3?. Applying equation (1A.3[) (k — 1) 

we get: 



W{u{X n+1 ) | jr n ) = £E>(X n+1 )I { £ =(ij . )} I T n ) (A.13) 
= ^E ¥ '(-u(X n+ i)I{ 6 )< n+ i}I{^ = (jj)} | jF n ) 
+ E </3 ( , u(X n+1 )I{ e ) >n+1 }I{^ = (jj)} | JF„) 

= E^n J k Uj [u(y)(l - U^(0,y,I^))f°l(yUdy) \ 
+E*(7 u(y)( % + Pii n«(0,y,n^))/^(X n+1 )M^) I JT„ 



We see that conditional distribution of X n+ \ given T n depends only on 
component of rj n what ends the proof. 



Lemma 9 Let 
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TQ k h(x 14+2 ,j,d),ifk>l, 
T ifk = 0. 



(A.14) 



Then, for function h(xi d+2 , 7, 5) given by ( [73]) and fc > 1, following equalities 
hold: 



Q k h ( x M+2 ,7,£) 

= max J ( 1 -Py + «j J] m 7i^ M+2 \ J ^ ~ s fc-ife,d+2, 7.<*) }> , 



max ■ 



Sk-i{x 24+3 ,j,po f% (x d+3 ) 06) \fJ,{dx d+3 ), 



where: 



So(z M +2, 7, S) = X] 1 - V% + fcj E ^Tjf^\ I ^ ~ 7ij)^A.15) 



Moreover for k > and vector r] n+ i = (X n _ dn+1 , II n+ i, B re+ i) ; function Sk 
has the property: 



o ( Y TT R ^ _ St(2C 1 -d,n+l i n B ,y°/l(^+l) O #n) , . , 
SfclAn-^n+l^Wli^n+lJ - g , y = w i • [A. lb) 



PROOF. Notice that lemmas U formulas ftATTOl) i flATTT]) enable us to 
rewrite function h{X_ n _ d n+l , n n+1 , 5 n+1 ) in the following way: 
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h ( Kn-d,n+l^n+l,B n+ i) (A.17) 

=e( 1 -4+ % E H^) (i - 

= E (i - 4 + « E I^wf^ ) f 1 - n M (o.i, +1 .n«)) 

i,j \ m=\Pij L 'Q Ul.re-d.n+lV 

xr'J(o,i nin+1 ,5 n ,n n ) 
_^/ r (i-pa-)p i3 -(i-nyW f0 ,^ y . 

i,j V ' — n,T»+l''°'" n ™J 

L m-l(^n-d,n) gij ( 1 ~ ) B^ ,l,j, y A 

Using definition of operator T, equation OA. 170 . for A; = and (2C n -i-dn; n n , S n ) 
fe,d+2>7J) we get 



s o(x.i,d+2i 7, <*) = E ¥ '(/i(X n _ djn , X n+1 , II n+ i, Bn+i)!^) (A.18) 

= / HKn-d,n^y^ u n+i, B n+ i)S(0, X n ,y, B n ,U n )fi(dy) 
Je 



2 



„ d+1 rij (y \ 

+E / ^ E gfe^ ^^ 1 - Wg^W 



1 Pij^O 1 ULn-d 



E (i - 4 + « E - W 



i,j \ m=l rij 

/ d+1 



E ( 1 " 4 + fti E t m r ^ r V+2 J - H^ir 



Hence, applying equations (IA. 1Q[) and flA.llj) one more time we end with 

i,j \ m=l Pij^Q (2L n -d+l,n+l) J $(0, X nn+1 , B n ,H n ) 

_ s 0{2Ln-d,n+li 

U n ,pof Xn (X n+1 )oB n ) 
S(0, 2Ln,n+li B n , n n ) 

If k — 1, then by definition of Q: 
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7, 5) = max jj^ ( 1 - pj + ^ J] 



1=1 Pij L J (%l,d+2\ 



(1-7^)<%,(A.19) 



so(=i,d +2 ,7>^)}- 



Now, for (2Cn-i-d,n> n n , £?„) = (x l d+2 ,7, 5), taking into account link between 
and n^- 7 as well as between -B^'ii and i?^- 7 given by flA.lQj) and flA.llj) . 
we get with the support of (1A.13I) : 



s l{x ltd+2 ,j,6) = W max{h(X n _ dn ,X n+ i,U n+1 ,B n+1 ), 

)} I Tn 



(A.20) 



s (X n _ d n ,y,U n ,p o f^ n (y) o B n ) 



S(0,X n ,y,B n ,U n ) 



5(0, X n , y, B n ,U n )fj,(dy) 



! maxi Wl-pJ + ^J] 

JE I tj \ m=l^ 



d+1 ri,j 



Basing on (1A.20I) with the help of flA.lQj) and (1A.11I) let us verify formula 
( BED : 



s l(2Cn-d,n+l) n n +l, 



f\~^ / 1 d i ^m(X_ n _ d+l n+1 ,y) \ 0i j - • ■ 



maxw I 1-pfj+qij E 

L i,j V m=l ^ L (2£„-d+l,n+l , 2/) 



m=l ^ij " 

so(X n+1 _ d , n+1 ,y,n n+1 ,p o f® n+i (y) oB n+1 ) }n(dy) 

y E max i El 1 ~ Py + 9«E 



7i=l Pij-C'o' 7 (^n-d+l.n+li y)y 



5(0, 2£n,n+l! -^ni 



'•J 



L%(x n __ dW1 ,y) w-(i-iiy)Po-^:(^)^: +1 (yw [ 

n,n+l' 

5„,n n ) 

To 



s l\2Ln—d,n+li 



5(0) 2Cn,n+l) -^n> II n ) 

Suppose that lemma [9] holds for some k > 1. We will show that equations 
characterizing Q fc+1 /i and s^+i are true and that condition (1A.16|) for Sk+i is 
satisfied. It follows from definition of operator Q fc+1 that: 
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(A.21) 



d+l 



d+2, 



max < 1 — r>-- + qa 



(1 - 7*i)* 



Given (X n _ 1 _ d n , II n , S n ) = (aZid+2,7, ^) and basing on inductive assumption 
we have also: 

s fe+ i(x M+2 ,7,I) = E ¥ ' max|/t(X n _ dn) X n+ i,n n+ i,B re+1 ), (A. 22) 

s k(X.n-d,n> ^n+l-i Hn+lj ^n+l)| I -^n 

v / _ d ^ ^(2U-d,n,y) \ g(t/h(i-ny 



= / max < 

Je 



Sk(X n _ dn ,y,U n ,po J Xn {y) oB r 



S(0,X n ,y,B n ,IL n )n(dy) 



max < 



S(0,X n ,y,B n ,U n ) 



m=l "ij 



S ,n n ,po/\(y)oB n ) J^dy) 

= / ™* (e f 1 -4 + e f:^ v+2,y) m + M^ - ^ 

^ [ V m=lPij L ' (X2,d+2,y)j 

Finally, using (1A.22l) we obtain: 



s k+l ( Aji-d.n+l) Hn+1, B 



n+1, 



max 



E !-4+%E 



J'm (^n-d+l,n+l5 2/) \ ,0,i 



1 (2C n -d+l,n+l) y) , 

Sfc(X n+1 _ d n+1 , y, II n+ i,p o f° Xn+1 (y)oB n+1 )] fj,(dy) 



n+V 



/maxi E(l-4'+%E 



i.l 



m=l PijLo {2L n -d+l,n+l > 2/)/ 
gfc(2C n +l-d,n+n2/. n n,P°/y„ +1 (y) ° P ° /x n (gw+l) ° gn) 

gfc+i(Ai w _ d;W+ i,n n ,po/^ n (x n+ i) o^ n ) 
^(Oj 2Cji,n+u -^i, n n ) 



X_ n ,n+1> B n , II n ) 

//(dy) 
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